psychologywikiaorg-20200213-history
Experimental design
The first statistician to consider a methodology for the design of experiments was Sir Ronald A. Fisher. He described how to test the hypothesis that a certain lady could distinguish by flavor alone whether the milk or the tea was first placed in the cup. While this sounds like a frivolous application, it allowed him to illustrate the most important means of experimental design: * Randomization - The process of making something random * Replication - repeating the creation of a phenomenon, so that the variability associated with the phenomenon can be estimated * Blocking - the arranging of experimental units in groups (blocks) which are similar to one another * Orthogonality - Means perpendicular, at right angles or statistically normal. * Use of factorial experiments instead of the one-factor-at-a-time method Analysis of the design of experiments was built on the foundation of the analysis of variance, a collection of models in which the observed variance is partitioned into components due to different factors which are estimated and/or tested. Some efficient designs for estimating several main effects simultaneously were found by Raj Chandra Bose and K. Kishen in 1940 at the Indian Statistical Institute, but remained little known until the Plackett-Burman designs were published in Biometrika in 1946. In 1950, Gertrude Mary Cox and William Cochran published the book Experimental Design which became the major reference work on the design of experiments for statisticians for years afterwards. Developments of the theory of linear models have encompassed and surpassed the cases that concerned early writers. Today, the theory rests on advanced topics in abstract algebra and combinatorics. As with all other branches of statistics, there is both classical and Bayesian experimental design. Example This example is attributed to Harold Hotelling in Herman Chernoff, Sequential Analysis and Optimal Design, SIAM Monograph, 1972.. Although very simple, it conveys at least some of the flavor of the subject. The weights of eight objects are to be measured using a pan balance that measures the difference between the weight of the objects in the two pans. Each measurement has a random error. The average error is zero; the standard deviations of the probability distribution of the errors is the same number σ on different weighings; and errors on different weighings are independent. Denote the true weights by : \theta_1, \dots, \theta_8.\, We consider two different experiments: * Weigh each object in one pan, with the other pan empty. Call the measured weight of the i''th object ''Xi'' for ''i = 1, ..., 8. * Do the eight weighings according to the following schedule and let Y''i'' be the measured difference for i'' = 1, ..., 8: :: \begin{matrix} & \mbox{left pan} & \mbox{right pan} \\ \mbox{1st weighing:} & 1\ 2\ 3\ 4\ 5\ 6\ 7\ 8 & \\ \mbox{2nd:} & 1\ 2\ 3\ 8\ & 4\ 5\ 6\ 7 \\ \mbox{3rd:} & 1\ 4\ 5\ 8\ & 2\ 3\ 6\ 7 \\ \mbox{4th:} & 1\ 6\ 7\ 8\ & 2\ 3\ 4\ 5 \\ \mbox{5th:} & 2\ 4\ 6\ 8\ & 1\ 3\ 5\ 7 \\ \mbox{6th:} & 2\ 5\ 7\ 8\ & 1\ 3\ 4\ 6 \\ \mbox{7th:} & 3\ 4\ 7\ 8\ & 1\ 2\ 5\ 6 \\ \mbox{8th:} & 3\ 5\ 6\ 8\ & 1\ 2\ 4\ 7 \end{matrix} : Then the estimated value of the weight θ1 is :: \widehat{\theta}_1 = \frac{Y_1 + Y_2 + Y_3 + Y_4 - Y_5 - Y_6 - Y_7 - Y_8}{8}. The question of design of experiments is: which experiment is better? The variance of the estimate ''X''1 of θ1 is σ2 if we use the first experiment. But if we use the second experiment, the variance of the estimate given above is σ2/8. Thus the second experiment gives us 8 times as much precision. Many problems of the design of experiments involve combinatorial designs, as in this example. Types of design Some of the most popular designs are sorted below, with the ones at the top being the most powerful at reducing observer-expectancy effect but also most expensive, and in some cases introducing ethical concerns. The ones at the bottom are the most affordable, and are frequently used earlier in the research cycle, to develop strong hypotheses worth testing with the more expensive research approaches. Experimental * Randomized controlled trial ** Double-blind ** Single-blind ** Non-blind * Nonrandomized controlled trial *Randomized database studies Nonexperimental * Cohort study ** Prospective studies ** Retrospective studies ** Nested cohort ** Time-trend study * Case-cohort study * Case-control study (case series) ** Nested case-control study * Cross-sectional study Descriptive * Community survey Ordering of conditions An important aspect of some experiment designs is the ordering of different experimental conditions. *A-B-C-D Experimental design Important considerations When choosing a study design, many factors must be taken into account. Different types of studies are subject to different types of bias. For example, '''recall bias' is likely to occur in cross-sectional or case-control studies where subjects are asked to recall exposure to risk factors. Subjects with the relevant condition (e.g. breast cancer) may be more likely to recall the relevant exposures that they had undergone (e.g. hormone replacement therapy) than subjects who don't have the condition. The ecological fallacy may occur when analyses are done on ecological (group-based) data rather than individual data. The nature of this type of analysis tends to overestimate the degree of association between variables. Other terms * A "retrospective study" looks at past behavior, while a "prospective study" looks at future behavior. * "Superiority trials" are designed to demonstrate that one treatment is more effective than another. * "Non-inferiority trials" are designed to demonstrate that a treatment is at least not appreciably worse than another. * "Equivalence trials" are designed to demonstrate that one treatment is as effective as another. * When using "parallel groups", each patient receives one treatment; in a "crossover study", each patient receives several treatments. * A longitudinal study studies a few subjects for a long period of time, while a cross-sectional study involves many subjects measured at once. See also * Animal models * Between groups design * Case study in psychology * Conjoint measurement * Clinical trial * Cohort analysis * Debriefing (experimental) * Design of experiments * Epidemiological methods * Experimental control * Experimental methods * Experiment volunteers * Experimentation * Followup studies * Hypothesis testing * Meta-analysis * Population * Psychometrics * Repeated measures * Research setting * Sampling (experimental) * Statistical analysis * Statistical variables * Test construction References External links * Epidemiologic.org Epidemiologic Inquiry online weblog for epidemiology researchers * Epidemiology Forum An epidemiology discussion and forum community to foster debates and collaborations in epidemiology * Some aspects of study design Tufts University web site * http://www2.truman.edu/polisci/design.htm Truman State University Political Science Research Design Handbook *Description of how to design experiments *Articles on Design of Experiments *Czitrom (1999) "One-Factor-at-a-Time Versus Designed Experiments", American Statistician, 53, 2. *SAS Examples for Experimental Design Category:Scientific method Category:Experimental design ----- Please copy and paste this prompt to other appropriate areas. Feel free to edit as necessary Instructions_for_archiving_academic_and_professional_materials Research design: Academic support materials * Research design: Academic: Lecture slides * Research design: Academic: Lecture notes * Research design: Academic: Lecture handouts * Research design: Academic: Multimedia materials * Research design: Academic: Other academic support materials